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campos de STEM, este estudio evalua la participation en clubes de matematicas influye en las 
calificaciones de matematicas y como la participacion en clubes de ciencias influye en las 
calificaciones de ciencias. Ademas, este estudio evalua como la participacion en clubes de 
matematicas o la ciencia se asocia con la probabilidad de seleccionar un campo STEM en la 
universidad. Utilizando los datos del Estudio Longitudinal Nacional de Salud Adolescente (Add 
Elealth) para examinar estas relaciones, los resultados sugieren que existe una brecha en el 
rendimiento y la persistencia STEM entre estudiantes que participan y no participan en actividades 
extraescolares relacionados con STEM. Mientras que, en su mayor parte, los resultados no fueron 
diferenciadas por sexo o raza / etnia en si, en realidad eran distinguibles por la situacion de pobreza 
y de la interaccion entre la raza y la condicion de pobreza. 

Palabras clave: club de matematicas, club de ciencias; STEM 

Participa§ao nos clubes de ciencia, tecnologia, engenharia e matematica (STEM) e 
desempenho escolar STEM 

Resumo: Para se ter uma compreensao mais solida da rela^ao entre as atividades 
extracurriculares realizadas nas areas de STEM nas escolas e o sucesso e persistencia dos 
alunos em areas STEM, este estudo avalia si a participa^ao em clubes de matematica 
influencias as qualificayoes em matematica e como participa^ao em clubes de ciencia 
influenciar qualifica^oes cientificas. Alem disso, este estudo avalia si a participa^ao em 
clubes de matematica ou de ciencias esta associada com a probabilidade de selecionar um 
campo STEM na faculdade. Usando dados do Estudo Nacional Longitudinal de Saude do 
Adolescente (Add Health) para examinar essas relates, os resultados sugerem que existe 
uma brecha no desempenho STEM e persistencia entre os alunos participantes e nao 
participantes em atividades extracurriculares relacionadas com STEM. Embora, na maioria 
das vezes, os resultados nao foram diferenciados por sexo ou ra^a /etnia em si eram 
realmente distinguiveis pelos indicadores de pobreza e a intera^ao entre raya e pobreza. 
Palavras-chave: clube, clube da matematica, ciencia, STEM 

Introduction 

The United States relies on scientists and engineers to ensure our national security, to solve 
our most critical problems, to further our body of knowledge, and to increase our general standard 
of living (Hira, 2010). Despite the essential role this workforce plays, however, policy makers, 
educators, and business leaders are deeply concerned about declining quantity and quality of 
American youth in line to adopt these positions. In terms of quantity, the percentage of students 
pursuing degrees in engineering, physical science, and math have remained stagnant, and the 
percentage pursuing computer science has declined (National Science Board, 2010). In terms of 
quality, American youth are ill-prepared compared to many of the countries we would consider as 
counterparts. For example, on the Programme for International Student Assessment (PISA) exams, 
15 year-olds in the United States rank 16 out of 26 countries in science literacy and 19 out of 26 
countries in mathematical literacy (National Science Board, 2010). These statistics do not bode well 
for the rising generation of the U.S. workforce who will enter into a highly competitive, global 
economy. 

Due to these concerns regarding both quantity and quality, there has been increased pressure 
on the education system to take a more proactive approach in preparing youth for careers in STEM 
fields. Indeed, decades of research findings and reports disseminated by academia, government 
agencies, and industry support the alarming conclusion that there is a serious deficit in our education 
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system (Rask, 2010); hence the need for an intervention particularly at the K-12 level is in order, 
such that the U.S. economy can sustain global competitiveness in the future (Augustine et al., 2010). 
In this context, there is an ever-growing research and policy interest geared at understanding how 
and why American youth develop, cultivate, and sustain an interest in science, technology, 
engineering, and mathematics (STEM)-related subjects. 

Moreover, data show that this national problem is not unique to any subgroup of the 
population: it is not localized to urban or suburban students, or to any particular gender, race, or 
socioeconomic group. Rather, the lack of high quality students in STEM is widespread among all 
youth in the U.S. (Augustine et al., 2010). Part of this pervasive problem can certainly be attributed 
to the lack of adequate formal preparation in K-12 grades (i.e., curriculum and classroom 
instmction). There are significant challenges that exist across the comprehensive K-12 education 
pipeline in STEM; a lack of appropriate resources, insufficient teacher training, and ineffective 
teacher pedagogy are often highlighted as K-12 factors inhibiting student preparation and 
performance in STEM (National Research Council, 2011). Prior research has called for a revamping 
of formal STEM education in order to more effectively prepare our youth. 

Along with formal preparation, non-formal exposure to STEM inside and outside of school 
(or lack thereof) has shown to be significant in a student’s trajectory into (or out of) a STEM-related 
career (Williams & Gottfried, 2010). Non-formal learning opportunities occur in planned, but highly 
adaptable ways in organizations or situations outside of formal or informal education experiences 
and often include activities such as academic clubs or career-related clubs (Butler & Serrell, 2001). 
Indeed, positive outcomes have been supported for those students participating in non-formal 
educational experiences: students’ achievement levels and social development improves based on 
students’ abilities to foster and cultivate intrinsic motivation and engagement in environments 
outside of the formal setting (Shernoff & Vandell, 2007). Other benefits of participation in non- 
formal activities have been reported, such as the development of positive social networks and 
leadership abilities (Broch, 2002; Lipscomb, 2007; Marsh, 1992). Furthermore, other benefits of 
participation in non-formal activities have been reported, such as the development of positive social 
networks and leadership abilities while facilitating involvement with new peer groups and adults, 
stimulating interest in a particular subject, and providing a richer experience than simply completing 
assignments in a formalized instructional setting (Broch, 2002; Lipscomb, 2007; Marsh, 1992). 

While limited, the empirical research in this area often focuses on the effects of non-formal 
in-school activities such as clubs. For instance, Eccles and Barber (1999) assessed the effects of 
participating in school-based academic clubs, such as foreign language club. Relying on a sample of 
approximately 1,300 predominantly White high school students in Michigan, the authors found that 
adolescents who indicated that they had participated in an academic club at school had higher than 
expected high school GPAs and were more likely to be enrolled in college at age 21 than were their 
peers who indicated that they did not participate in extracurricular clubs at school. It should be 
noted that the study relied on general measures of extracurricular activities and GPA and did not 
evaluate particular STEM fields in looking at the effect of non-formal participation. 

That said, however, other research has looked specifically at the effect of club participation 
on STEM outcomes under the premise that students’ involvement in school-based extracurricular 
activities develops their self-confidence, ability to persist in competitive situations, and focus among 
other characteristics - all of which may be related to improved interest and performance in STEM 
(Csikszentmihalyi, 1996; National Federation of State High School Associations, 2005; Vandell, 
Pierece, & Dadisman 2005;). In studying the effects of extracurricular club participation on STEM 
achievement, Lipscomb (2007) found that students at the secondary level who participated in 
athletics or extracurricular clubs exhibited a 2-percent increase in math and science test scores and 1- 
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percent increase in math test scores, respectively. The study utilized the National Education 
Longitudinal Study (NELS) of 1988 and included a base sample of 16,305 students. 

The nationally-representative findings in Lipscomb’s (2007) study built upon earlier work by 
Camp (1990), which also found a positive correlation between extracurricular student activity and 
academic achievement. In fact, Camp’s (1990) conclusions validated even earlier research as well: 
that participation in in-school student clubs had a positive prediction on academic achievement 
(Holland & Andre, 1987). It should be noted, however, that these studies utilized general measures 
of participation in extracurricular school clubs; none were domain specific, such as the influence of 
math club participation on math achievement. Although there have been several decades of research 
evaluating a broad relationship between extracurricular club participation and achievement, it is 
necessary to take a more domain-specific approach in examining the relationship between 
participation in math and/or science clubs and STEM outcomes. Doing so will allow for further 
clarity into the educational pipeline as to how exposure in specific STEM fields relates to success 
and persistence in those particular subject areas. 

Prior work has suggested that the positive outcomes related to participation in non-formal 
STEM experiences, such as in-school clubs, reflects combination of the skills students learn in these 
activities as well as the higher motivation of those who select into those clubs in the first place. 

Thus, it is important to separate the components of this combination in order to develop a more 
robust understanding of the tme effects that non-formal, school-based STEM activities may have on 
building skills and promoting both success and persistence in STEM subjects. This study takes this 
next step by evaluating the relationship between participation in math or science club and STEM 
educational attainment in those respective, fields. Given these issues, there are three research 
questions: 

1. Does in-school extracurricular math and science club participation have a positive 
association with math and science schooling success, respectively? 

2. Does gender, race, or socioeconomic status moderate the relationship of of math or 
science club participation on STEM schooling success? 

3. Does in-school extracurricular math and science club participation have a positive 
association with an increased probably of declaring a STEM major in college, respectively? 

In addressing these questions, this study makes three meaningful contributions. First, prior 
studies have examined the relationship between participation in extracurricular school activities and 
school success (e.g.. Camp, 1990; Dpscomb, 2007) with some research examining math and science 
outcomes. However, no study has evaluated the relationship between a domain-specific 
extracurricular club on STEM achievement. That is, no study has examined the relationship between 
math club participation and math achievement or between science club participation and science 
achievement. This study does both. Additionally, this study evaluates if a long-term relationship 
persists between math or science club participation by looking not simply at high school STEM 
success but also at the differences in the probability of graduating with a STEM major in college for 
those students participating in math or science club. 

Second, this study asks these research questions while also employing a wider range of 
contextual variables than those that have been used in prior research, such as student academic 
attributes, official student transcript and course-taking data, and school characteristics. The analyses 
in this research study include these additional variables. 

Third, an additional gap in earlier research is that there is only little examination of the 
heterogeneity of these relationships by gender, race, or socioeconomic status (SES). However, there 
is a higher concentration among minority students who demonstrate a deeper lack of interest in 
STEM. According to Rowe (1977), the value of early and extensive exposure to science influences 
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decisions that Black students make about science professions. This is, of course, not unique to just 
Black students and can be applied to many student populations where research shows that minority 
ethnic groups have similar attitudes and aspirations or lack thereof for STEM careers (Crisp et al., 
2009). However, particular attention can be drawn to students of color given the fact that they lag 
behind their white counterparts in STEM across the board in terms of preparation and exposure to 
STEM (Simpson, 2001). Prior research suggests disparities in STEM fields by poverty versus non¬ 
poverty students, though these academic disparities were not necessarily limited to STEM 
(Donaldson et al., 2008). Hence, poverty students may have less involvement in STEM, lower 
achievements in these fields, and a lower rate of participation compared to non-poverty students. 
And looking at gender, there is evidence to suggest that differences in science-related experiences at 
the formal level extend into informal learning environments, where girls are less likely to engage in 
out-of-school activities than boys. Girls potentially miss out on the many kinds of skills developed in 
these non-formal activities (e.g. science fairs) where they explore, assemble, and tinker in a way that 
enhances interest and success in STEM (Greenfield, 1996; Jones, 1997; Rennie, 1987). If there truly 
is an association of math or science club participation with STEM outcomes, one might suspect that 
those of different demographic backgrounds would receive differential benefits from math or 
science club participation. 


Method 


Data 

As the purpose of this study was to determine the presence of a relationship between math 
or science club participation and math or science schooling success and on the probability of 
choosing a STEM major in college, this study utilized data from the National Longitudinal Study of 
Adolescent Health (‘Add Health’). Data collection was initiated in 1994 to examine how adolescents’ 
social contexts influence schooling and health outcomes. The dataset contained thousands of 
variables on adolescents’ families, schools, neighborhoods, and peers. The first wave, with in-school 
surveys, in-home surveys, parent surveys, and school administrator surveys, was conducted with 7 th 
to 12 th grade students between 1994 and 1995. The second wave, with in-home surveys of the 
students and school administrator surveys was conducted one-year later in 1996. The third wave, 
with an in-home survey for the students in the sample, was conducted in 2001-2002 when 
respondents were 18-26 years old. Additional information regarding the add health dataset is found 
in Harris et al. (2009). 

It is crucial to note that the research objective in this study was to ascertain whether a STEM 
experiences in high school associate with both high school and college outcomes. That is to say, the 
research questions were framed to have long-term implications, and so by design they required 
longitudinal data. To truly conduct a longitudinal study, it was important to observe youth over time. 
With these parameters in mind. Add Health was the most recently available national data to address 
the questions posed with the contextual variables that were required for the model. There was still 
much to be gleaned from these data that are of relevance to the current policy and research dialogue 
on these issues, even with information on students attending high school in the late 1990’s. 

Analytical Samples 

This study used Add Health data to create two samples. The first sample was utilized to 
generate the probability of participating in a math or science club in school, as it will be described 
below. This sample was based on having been a student in the in-home Wave I survey, and data 
must be available as to whether or not each student had participated in a math or science club at 
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school (N = 15,356). Math or science club was determined by a survey question in Wave I, in which 
a student was asked to indicate school club participation from a list of possibilities. Of the 
possibilities, “math club” and “science club” were both options. Thirty-one additional clubs were 
listed, though none could be classified as having a STEM focus (e.g., French club). Note that no 
additional detail was provided regarding math and science clubs. This is addressed in the limitations 
section of this article. 

The second sample was utilized to examine the effect of math or science club participation 
on math and science GPA or on the probability of choosing a STEM major in college. To be 
included in this sample, students must have had non-missing data on math and science GPA or on 
major selection in college in addition to the non-missing observations used to predict math or 
science club participation in the first sample. This final regression sample, as described in Table 1, 
included a total of N = 3,223 students. Within this sample, 202 students participated in science or 
math clubs in school, and the remainder did not. 

Variables 

Table 1 presents the means and standard deviation values of the dependent and independent 
variables used in this analysis, as broken out by whether or not a student participated in math or 
science club (i.e., STEM club in the table). A test of mean differences was employed to determine if 
the two student groups - participants versus non-participants - differed in a statistically meaningful 
way across the sample. Participation in a STEM club was binary. It indicates if a student had 
participated in a math or science club at school in the academic year in which the Wave I survey was 
administered. 

There were two sets of dependent variables in this analysis. The first were math and science 
high school GPAs. To supplement the educational data in Add Health, the Adolescent Health and 
Academic Achievement (AHAA) study began in 2001 with the collection of official high school 
transcripts. Researchers collected transcripts for 12,237 Add Health subjects based on the more than 
1,200 schools they last attended. This investigation used students’ overall math and science GPAs as 
calculated by AHAA researchers based on transcript data. Table 1 suggests that there were 
statistically significant differences between the math and science GPAs of those students 
participating versus not participating in math and/or science clubs in school. These differences were 
tested at the p = 0.05 level. Hence, this difference of means served as a benchmark result of the 
influences of math or science club participation and merited further exploration with the use of 
more rigorous analytical techniques. 

The second dependent variable was having a STEM major in college. The variable employed 
here was binary: a 1 indicates if a student declared a STEM major in college, and a 0 indicates 
otherwise. Students could identify more than one major in college in the survey. Thus, if a student 
graduated with a STEM major in addition to a second major, he or she was also identified as having 
a STEM major. The determination of those college majors deemed as belonging to STEM was 
executed for this study based on the academic classification codes assigned to each major in the 
database. Majors that fell into science, technology, engineering, mathematics, or a hybrid of these 
four disciplines were considered to be STEM majors. A list of STEM majors included in the dataset 
is available upon request. For this dependent variable, Table 1 presents mean differences between 
those who participated in math or science club and those who did not. The results suggested that 
students in math or science clubs tended to have a greater chance of having a STEM major in 
college. 
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Table 1 

Descriptive Statistics _ 

STEM Club 


Full Sample Participants Non-Participants 



Mean 

Std Dev 

Mean 

Std Dev 

Mean 

Std Dev 

Outcomes 







High school math GPA 

2.23 

0.92 

2.606) 

0.97 

2.21 

0.92 

High school science GPA 

2.33 

0.93 

2.766) 

0.94 

2.30 

0.95 

Selection of STEM major in college 

0.32 

0.47 

0.516) 

0.50 

0.30 

0.46 

Student demographic information 







Male 

0.47 

0.50 

0.47 

0.50 

0.47 

0.50 

Minority 

0.37 

0.48 

0.38 

0.49 

0.37 

0.48 

Asian 

Middle school (versus high school) 

0.15 

0.36 

0.17 

0.38 

0.15 

0.36 

identifier 

0.26 

0.44 

0.35 

0.48 

0.26 

0.44 

Household income (in thousands) 

46.90 

44.03 

44.32 

39.28 

47.07 

44.34 

STEM preparation and motivation 







Highest math course taken in high school 
(1 = lowest, 9 = highest) 

Highest science course taken in 

6.28 

1.82 

7.146) 

1.75 

6.22 

1.81 

high school (1 = lowest, 6 = highest) 
Percentage of failed math semesters in 

4.48 

1.26 

4.956) 

1.22 

4.45 

1.25 

high school 

Percentage of failed science semesters in 

0.11 

0.19 

0.08 

0.17 

0.11 

0.19 

high school 

0.08 

0.19 

0.06 

0.17 

0.09 

0.19 

Peabody PVT test score 

Scale of desire to attend college 

102.58 

14.09 

105.496) 

13.40 

102.38 

14.11 

(1 = lowest, 5 = highest) 

Parents are interested in child's grades 

4.55 

0.91 

4.886) 

0.48 

4.53 

0.93 

(0 = lowest, 1 = highest) 

0.92 

0.27 

0.91 

0.29 

0.92 

0.27 

School information 







Private 

0.08 

0.27 

0.08 

0.27 

0.08 

0.27 

Magnet 

0.37 

0.48 

0.40 

0.49 

0.37 

0.48 

Urban 

0.30 

0.46 

0.39 

0.49 

0.29 

0.45 

Suburban 

0.52 

0.50 

0.47 

0.50 

0.52 

0.50 

n 

3,222 


202 


3,020 



Note: (a) Statistically different from the non-participant sample at p<0.05 


Sixteen independent variables were employed in this analysis. The covariates used as control 
variables in the regression models to predict math and science GPA or STEM major selection are 
the same covariates that were first used to estimate a student’s probability of participating in a math 
and/or science club, as described in the analytic strategy to follow. Of these independent variables, 
this study employed a standard set of demographic variables that has been used to predict high 
school GPA when using Add Health data (Gottfried & Polikoff, 2012) or STEM major selection 



Education Policy Analysis Archives Vol. 21 No. 79 


(Williams & Gottfried, 2010). In general, Table 1 did not show any statistically significant differences 
between student demographics between participants and non-participants in math and/or science 
clubs. 

A second set of variables fell into a category described by Williams & Gottfried (2010) as 
‘preparation and motivation” for the pursuit and persistence in STEM. First, to account for 
preparation in STEM, this study evaluated the highest course completed while in high school in 
math and science. The scale of this measure was 1, being an introductory course, to 9 being an 
extremely advanced course for math and 1 to 6 for science. The subject categories of the math 
course sequence included: 1, Basic/Remedial Math; 2, General/Applied Math; 3, Pre-algebra; 4, 
Algebra I; 5, Geometry; 6, Algebra II; 7, Advanced Math (Algebra III, Finite Math, Statistics); 8, Pre¬ 
calculus (includes Trigonometry); and 9, Calculus. The subject categories for the science course 
sequence included: 1, Basic/Remedial Science; 2, General/Earth Science; 3, Biology I; 4, Chemistry; 
5, Advanced Science (Biology II, Chemistry II); and 6, Physics. These categories reflected a hierarchy 
of courses ranging from less to more advanced. Note that students did not have to pass through 
each category of the sequence. For instance, students might have taken either Advanced Math or Pre 
calculus, but not both. Additionally, while most students’ course-taking patterns reflected a linear 
movement through the sequence, a minority of students may have had different patterns (i.e., 
Chemistry may not always precede Physics). Students who participated in math or science clubs tend 
to have taken courses higher in the sequence in these subjects, as presented in Table 1. A math and 
science failure index was also included in this analysis as the percentage of semesters failed in math 
or science over the course of middle or high school. 

Additional preparation in STEM is measured by individual scores on the Add Elealth Picture 
Vocabulary Test (PVT) (Williams & Gottfried, 2010). The test was an 87-item, multiple-choice, 
computerized, abridged version of the Peabody Picture Vocabulary Test administered to students at 
the start of the Wave I survey. Raw scores were standardized by age. Students in math or science 
clubs had statistically higher PVT scores than do students who did not participate. 

Motivation for STEM persistence (and general schoolwork) in this study, as consistent with 
the framework put forth by Williams & Gottfried (2010), was measured in two ways. First is a scale 
indicating a student’s desire to attend college was defined by his or her Wave I response to the 
question of how much he or she wanted to attend college on scale of 1 to 5, with 1 = low and 5 = 
high. Students who participated in math or science clubs had statistically higher desires to go to 
college, as presented in Table 1. Second was a student’s interpretation of parental interest in his or 
her school life: a binary variable indicts if a student felt that a parent was interested in his or her 
academic grades. 

Finally, the analysis included a set of school-level predictors. These included indicators for 
whether the school that a student attends was private, magnet, urban, or suburban. There were no 
statistically significant differences between the types of schools that participants in math or science 
clubs and non-participants attend. 

Analytic Strategy 

Baseline model. In the baseline empirical model of educational outcomes, student 
achievement was described using a linear relationship with a particular academic measure as the 
dependent variable and a vector of independent variables. In the described formulation of the model 
to follow, math or science GPA served as a dependent variable and STEM-club participation, 
student characteristics, and school characteristics serve as independent variables. Note that when 
STEM major selection was utilized as an outcome, the model became a probability model (i.e., 
logistic regression) rather than a linear model presented below. Regardless, the independent variables 
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remained the same whether GPA or STEM major selection was chosen as an outcome in the 
specification. This baseline model is displayed in equation (1), 

Y i g k = o + iSTEM it + 2 I igk + 2 F, + 3 S k + e igk (1) 

where Y was either math or science GPA for student i in grade^in school k; STEM indicated if a 
student participated in a math or science club at school k (whether math or science club was selected 
depends on whether math GPA or science GPA was the outcome, respectively); I was a vector of 
individual-level characteristics; F were student-specific family characteristics; and S included school 
characteristics. Finally, £ igk was a random error capturing individual variations over time as well as a 

school-specific random component that is common to all members of the same school. 

The regression presented as equation (1) contained a large span of individual, family, and 
school characteristics, and it allowed for a baseline analysis as to whether there was a systematic 
relationship between math or science club participation at school and math or science GPA, 
respectively. Elowever, estimates of 1, the coefficient on math or science club participation, may 
still have been biased under ordinary least squares methods. The reason was that there were current 
and past unobservable factors that could influence both math or science club participation (i.e., the 
independent variable) as well as math or science GPA (i.e., the dependent variable). For instance, 
student motivation might simultaneously be related to selecting into math club as well as a higher 
math GPA. As a consequence, the results from equation (1) served strictly as a baseline set of 
estimates; the results of a more rigorous methodology were compared to this first set of results. 

Strengthening the baseline model. A baseline model could test for whether there may be 
correlational evidence that students were acquiring non-formal STEM exposure from math or 
science club participation that was positively related to STEM schooling success or STEM major 
selection. Elowever, it may have been possible that students who already had abilities correlated with 
greater STEM success were participating in math or science clubs, though this was not possible to 
determine with the secondary dataset utilized in this study. It would have been ideal to randomly 
assign students into math or science clubs at school. Elowever, quasi-experimental methods 
provided an approximation in the absence of random assignment, and one widely accepted way in 
which quasi-experimental designs can emulate random assignment is through propensity score 
matching (Becker & Ichino, 2002; Schneider et al., 2007). 

This approach, which was adopted in this study, allowed for quasi-experimental contrasts 
between students in naturally occurring treatment and control groups, but who displayed similar 
likelihoods of experiencing the treatment based on their observed characteristics. In this study, 
propensity score matching was employed to contrast the math and science GPAs of students who 
did and did not participate in math or science clubs but who nonetheless displayed similar 
propensities to participate in them. Based on using a restricted comparison sample, the results from 
employing propensity score matching yielded an average treatment effect, which is considered a 
more robust finding than the average estimate derived from the entire sample in the baseline model 
(Eleckman, 1996). 

In general, propensity score matching is a two-stage process. It was conducted twice in this 
study: once for math club when math GPA was an outcome, and once for science club when science 
GPA was an outcome. In the first stage, a propensity score, or probability, was calculated for 
selection into the treatment group. In this case, the treatment was a student’s participation in a math 
or science club at school. The calculation of a propensity score was based on Rosenbaum and Rubin 
(1983). The propensity score P(Z) was defined as the conditional probability that a student with a set 
of Z observable characteristics will be in math (or science) club (C=l), i.e., P(Z) = Pr(C=l Z). The 
Z characteristics that are used to model the selection process into math or science club participation 
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were included based on prior literature examining what types of students typically succeed in STEM 
(e.g., Williams & Gottfried, 2010). 

The actual propensity score for each student was estimated using a logistic regression model 
(i.e., binary outcome model asking if a student did or did not participate in a STEM club) 
(Rosenbaum & Rubin, 1984): 

log[P(Z)/(l-P(Z))] = + XZ), 

where and were parameters and^Z) was the specified function. Again, this model was 
conducted twice - once for math club and once for science club. The propensities were evaluated 
separately depending on whether math GPA or science GPA is an outcome. 

To populate this function, this study employed 16 covariates to model a student’s propensity 
to participate in a math or science club - these were the same covariates and interaction terms that 
were used as control variables in the baseline model. Variables were kept in the model if they were 
statistically significant at the conventional level and/or if theory suggested a strong relationship 
between a particular variable and math or science club participation. Indeed, the covariates 
predicting math or science club participation were identical, and all 16 covariates were employed in 
both models. 

The second stage of propensity score matching entailed using the estimated propensity 
scores from the first stage to match students who were and were not in math or science clubs at 
school in order to derive the average treatment effect. Three different matching mechanisms are 
employed in this study as a way of conducting a sensitivity analysis (Luellen et al., 2005). The first 
was matching by blocks, in which treatment and control students are stratified into different groups 
by having extremely similar propensity scores. Within each block, the average treatment effect was 
calculated, and then they were averaged across all blocks to produce an overall average treatment 
effect - i.e., a more refined estimate of the relationship between math or science club participation 
and STEM outcomes than what equation (1) might have suggested. 

The second method for determining the average treatment effect was nearest-neighbor 
matching. Elere, treatment students and control students were paired based on the close values of 
their propensity scores. Nearest-neighbor matching with replacement was employed, because it 
allowed for control students to be matched to more than one treatment student. This enabled for 
flexibility in that many control students might have matched to multiple treatment students. 

Finally, kernel matching was employed. Elere, individual students in the treatment group (i.e., 
those who participated in STEM clubs) were matched to a weighted mean of control cases. The 
weights were determined by the distance of each propensity score to the propensity score of the 
treatment case to which each control was being matched. With kernel matching, the analysis was 
bootstrapped with 1,000 repetitions and a bandwidth of 0.01, as consistent with prior research 
(Pagan & Ullah, 1999). 

Overall, the estimated coefficient on math or science club participation represented the 
average treatment effect of participation on a student’s STEM schooling outcomes. If the estimated 
value of this coefficient were statistically significant and positive, this indicated that students who 
participated in math or science clubs tended to have higher schooling success than students who did 
not participate in math or science clubs. Relying on propensity score matching thus enabled this 
study to hypothesize on the direction of any bias from having relied strictly on a baseline regression 
model. 


Results 


Baseline Results 
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Table 2 presents the results of the regressions for the relationship between math club 
participation on high school math GPA and also for the prediction of science club participation on 
science GPA. Under the first model of each subject, the prediction of math or science club was 
conducted without any other independent variable included in the model. Under the second model 
of each subject, the results are presented from employing a regression using the same covariates that 
were also used to estimate a student’s propensity to participate in math or science club in the 
matching algorithm. That is, the second model of each table displays the differences in GPA of 
students who did and did not participate in math and/or science clubs, after statistically controlling 
for the covariates used to predict a student’s propensity to participate in a club. Note that in both 
subjects, students have not yet been matched on their propensity to be in math or science clubs. 

The key variable in this study is presented in either the first or second row of the table, 
depending on the subject area of the outcome. The table indicates that participation in math club 
had a positive relationship with math GPA, and participation in science club had a positive 
relationship with science GPA. This was true whether participation is examined in the first or 
second models within each subject area. The inclusion of a full set of covariates, however, did 
reduce club participation’s positive effect in both subject areas from 0.47 to 0.25 GPA points in 
math and 0.42 points and 0.25 GPA points in science, though the coefficients remained significant 
at the p<0.001 level across all models. As such, though there was a reduction in the size of the 
coefficients from the inclusion of a full span of covariates, it did not alter this study’s fundamental 
premise that non-formal STEM exposure positively correlated with STEM schooling success. 

The use of a variant on the standardized beta coefficients is often implemented as the effect 
size when the independent variable of interest is binary (e.g., Datar, 2006). The effect sizes of 

participating in math or science club, as defined by the standardized regression coefficient X! O y , 
were 0.50 and 0.27 for models 1 and 2 in math, respectively, and 0.45 and 0.27 for models 1 
and 2 in science. This implies that there existed a medium-sized (Cohen, 1988) standardized positive 
relationship between math or science club participation and respective GPA, though the relationship 
was tempered by the inclusion of a full set of covariates. 
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Table 2 

Regression Results for Math and Science Cumulative GPA 


Predicting Math GPA Predicting Science GPA 


Model 1 


Model 2 

Model 1 

Model 2 

Coefficient 

Std 

Error 

Std 

Coefficient Error 

Std 

Coefficient Error 

Std 

Coefficient Error 


Participation in math club 

0.469”* 0.088 0.228”* 

0.057 

0.415”* 0.084 

0.250”* 

0.053 

Participation in science club 






Student demographic information 






Male 

-0.132”* 

0.021 


-0.142”* 

0.021 

Minority 

-0.216”* 

0.024 


-0.184*** 

0.024 

Asian 

0.116”* 

0.030 


0.091” 

0.030 

Middle school (versus high school) identifier 

0.021 

0.024 


0.046+ 

0.024 

Household income (in thousands) 

0.000 

0.000 


0.000 

0.000 

STEM preparation and motivation 






Highest math course taken in high school (1 = lowest, 9 = highest) 

0.124”* 

0.008 


0.135”* 

0.008 

Highest science course taken in high school (1 = lowest, 6 = highest) 

0.050”* 

0.011 


0.048*** 

0.011 

Percentage of failed math semesters in high school 

-2.571”* 

0.072 


-0.666”* 

0.072 

Percentage of failed science semesters in high school 

-0.099 

0.071 


-2.107”* 

0.070 

Peabody PVT test score 

0.003” 

0.001 


0.006”* 

0.001 

Scale of desire to attend college (1 = lowest, 5 = highest) 

0.006 

0.012 


0.021 + 

0.012 

Parents are interested in child's grades (0 = lowest, 1 = highest) 

-0.022 

0.038 


-0.005 

0.038 

School information 






Private 

0.122” 

0.043 


0.009+ 

0.043 

Magnet 

0.006 

0.023 


0.041 

0.023 

Urban 

-0.047 

0.034 


0.041 

0.034 

Suburban 

-0.014 

0.029 


0.023 

0.029 

n 

3,220 3,220 


3,220 

3,220 


R 2 

0.01 0.59 


0.01 

0.61 



Notes: **p<0.001; **p<0.01; *p<0.05; + p<0.10 
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Briefly turning to the set of control variables employed in this study, gender and race had 
negative relationships with high school math and science GPA, holding all else equal. This result was 
consistent across both subject areas and is also consistent with much of the literature (Argys, Rees, 

& Brewer, 1996; Caldas & Bankston III, 1997; Ogbu, 1989; Summers & Wolfe, 1977). The results 
for preparation and motivation were consistent with prior work in their relationships with STEM 
outcomes (e.g., Williams & Gottfried, 2010): taking increasingly more advanced courses in the 
sequence was related to higher GPAs, for instance. Finally, there was a general lack of significance in 
the relationships between school-level covariates and math and science GPA. 

Accounting for Unobserved Heterogeneity 

The results from the baseline model suggested that there may be evidence that students were 
acquiring non-formal STEM exposure from math or science club participation that was positively 
related to STEM schooling success. However, it might have been possible that students who already 
had abilities correlated with greater STEM success (i.e., through preparation, motivation, and 
exposure) might have participated in math or science clubs at school. 

Table 3 provides a more robust version of the analyses presented in Table 2: it displays the 
main results for math club participation on math GPA for both baseline analyses and propensity 
score matching models. The first two models were the results from the math models in Table 2. 

New to the table are the three latter models, which employed propensity score matching based on 
block matching, nearest neighbor matching, and kernel matching, as described in the Method 
section above. When matched on their propensity to participate in a math club, students in math 
club had higher math GPAs, as evidenced by both block matching and kernel matching. Even after 
controlling for a more rigorous regression model with the use of propensity score matching, 
participation in math club significantly related to math GPA. 

Table 3 

Estimated Effects of Math Club Participation of Cumulative Math GPA _ 

OLS Regression (Table 2) Propensity Matching 

Block Nearest Kernel 

No covariates Full model Matching Neighbor Matching 

Math GPA 0.469*** 0.228*** 0.233* 0.136 0.426*** 

_ 0.088 0.057 _ 0.104 0.093 0.089 

Notes: ***p<0.001; **p<0.01; *p<0.05;+p<0.10 
Standard errors are in italics under the each estimate 

Table 4 presents similar findings for the relationship between science club participation and 
science GPA. Analogous to Table 3, the first two models present the results from the science 
models in Table 2. New to Table 4 are the three later models, which employed propensity score 
matching based on block matching, nearest neighbor matching, and kernel matching. When matched 
on their propensities to select into a science club, students in science club had higher science GPAs, 
as evidenced by significant coefficients in all three propensity score matching methods. In other 
words, the participation in science club is related to statistically significant differences in science 
GPA. 
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Table 4 

Estimated Effects of Math Club Participation of Cumulative Science GPA _ 

OLS Regression (Table 2) Propensity Matching 

Block Nearest Kernel 

No covariates Full model Matching Neighbor Matching 

Science GPA 0.415*“ 0.250*“ 0.251* 0.212 + 0.396*“ 

_ 0.084 0.053 _ 0.088 0.090 0.080 

Notes: ***p<0.001; **p<0.01; *p<0.05;+ p<0.10 
Standard errors are in italics under the each estimate 

Results by Student Characteristic 

Tables 5 and 6 disaggregate the analyses in Tables 3 and 4, respectively, to determine if 
different demographic groups had different relationships to GPA by participation in math or science 
club. Each cell represents the coefficient on math club participation when math GPA was utilized as 
an outcome in Table 5 or science club participation when science GPA was utilized as an outcome 
in Table 6. Each cell represents the results from fully interacted models based on the demographic 
characteristic indicated in the leftmost model. 

Table 5 suggests that for math GPA, the relationship between math club participation and 
math GPA was approximately the same for males and females- the coefficients in each group 
tended to be fairly similar to one another across OLS regression and propensity score matching 
models. The same was true for students across different race/ethnicity categories. That is, examining 
across each racial/ethnic group shows that the results did not differ greatly whether a student was 
Black or Hispanic, Asian, or White. A major distinction, however, arose between poverty and non¬ 
poverty students (where poverty was previously determined in the dataset by being in a family that 
did or did not receive welfare assistance). There was an overall lack of statistical significance on the 
coefficients for poverty students. This was tme for OLS regressions and propensity score models. 

On the other hand, non-poverty students who participated in math club also have higher math 
GPAs. 

This result warranted further investigation, and as such, poverty versus non-poverty status 
was delineated by race. Both Black or Hispanic and White non-poverty student subsamples tended 
to have higher math GPAs when participating in math club. On the other hand, participation in 
math club was not related to math GPA for poverty students across the same racial/ethnic groups. 
Asian students, regardless of poverty status, tended to have higher GPAs when participating in math 
clubs, however. It was noted that the high values of the effects in the Asian/poverty category may 
have arisen from the small sample size in this subgroup. 

Table 6, which presents the evaluation of the relationship between science club participation 
and science GPA, portrays a slightly different interpretation; this underscores the importance of 
examining club participation by subject area rather than as an aggregate measure. Generally speaking, 
the analysis for males and females looked fairly similar to what Table 5 presented for math club. 
Gender, once again, did not seem to moderate the association of science club participation on 
science GPA. Also as consistent with the analysis of math club participation in Table 5, there was no 
difference for White and Black or Hispanic students. However, for science club participation, there 
was no longer a statistically significant relationship for Asian students. 
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Table 5 


Estimated Effects of Math Club Participation of Cumulative Math GPA _ 

OLS Regression Propensity Matching 



No 


Block 

Nearest 

Kernel 


covariates 

Full model 

Matching 

Neighbor 

Matching 

Male 

0.505“* 

0.222“ 

0.336* 

0.327* 

0.441“ 


0.129 

0.084 

0.123 

0.132 

0.139 

Female 

0.434“* 

0.239“ 

0.314* 

0.328* 

0.411“ 


0.117 

0.078 

0.118 

0.118 

0.123 

Minority 

0.556“* 

0.327“* 

0.444“ 

0.421“ 

0.530“ 


0.137 

0.089 

0.156 

0.138 

0.162 

Asian 

0.532“ 

0.314“ 

0.297 

0.478* 

0.421* 


0.183 

0.108 

0.194 

0.177 

0.149 

White 

0.468“* 

0.150+ 

0.313“ 

0.548*** 

0.439*** 


0.118 

0.079 

0.100 

0.116 

0.093 

Poverty 

0.442 

0.314+ 

0.211 

-0.069 

0.375 


0.299 

0.183 

0.416 

0.286 

0.369 

Non-poverty 

0.483“* 

0.211“* 

0.354“ 

0.404*** 

0.442“ 


0.092 

0.060 

0.098 

0.098 

0.112 

Minority & poverty 

0.290 

0.365 

0.163 

0.322 

0.278 


0.427 

0.260 

0.771 

0.341 

0.400 

Minority & non-poverty 

0.599“* 

0.305“ 

0.493“ 

0.434*** 

0.571*** 


0.144 

0.096 

0.161 

0.148 

0.126 

Asian & poverty 

1.731* 

0.462 

1.281“ 

2.072*** 

1.458*** 


0.693 

0.513 

0.365 

0.258 

0.244 

Asian & non-poverty 

0.446“ 

0.294“ 

0.217 

0.409* 

0.346+ 


0.188 

0.113 

0.206 

0.152 

0.198 

White & poverty 

0.283 

0.192 

0.357 

0.532 

0.538 


0.456 

0.273 

0.645 

0.340 

0.509 

White & non-poverty 

0.418“* 

0.130 

0.319+ 

0.531*** 

0.443*** 


0.131 

0.083 

0.110 

0.127 

0.109 


Notes: ***p<0.001; **p<0.01; *p<0.05;+ p<0.10 
Standard errors are in italics under the each estimate 
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Estimated Effects of Math Club Participation of Cumulative Science GPA _ 

OLS Regression Propensity Matching 



No 


Block 

Nearest 

Kernel 


covariates 

Full model 

Matching 

Neighbor 

Matching 

Male 

0.367** 

0.224** 

0.193* 

0.177 

0.347** 


0.120 

0.074 

0.092 

0.122 

0.111 

Female 

0.482*** 

0.285*** 

0.321* 

0.252* 

0.464*** 


0.116 

0.076 

0.113 

0.125 

0.112 

Minority 

0.500*** 

0.328*** 

0.364* 

0.405** 

0.488** 


0.124 

0.081 

0.139 

0.127 

0.143 

Asian 

0.300 

0.116 

0.021 

0.015 

0.264 


0.217 

0.134 

0.192 

0.184 

0.182 

White 

0.445*** 

0.219** 

0.259* 

0.305* 

0.426*** 


0.107 

0.069 

0.116 

0.115 

0.104 

Poverty 

0.357 

-0.079 

0.275 

0.469* 

0.363 


0.328 

0.187 

0.237 

0.203 

0.244 

Non-poverty 

0.419*** 

0.277*** 

0.252* 

0.254* 

0.400** 


0.086 

0.055 

0.093 

0.093 

0.108 

Minority & poverty 

0.787 

0.258 

0.652 

0.849*** 

0.798* 


0.483 

0.255 

0.411 

0.191 

0.278 

Minority & non-poverty 

0.465*** 

0.342*** 

0.335* 

0.326* 

0.453** 


0.128 

0.086 

0.137 

0.140 

0.141 

Asian & poverty 

1.540 

0.438 

0.978** 

\ 971 *** 

1.521*** 


0.902 

0.659 

0.269 

0.490 

0.151 

Asian & non-poverty 

0.229 

0.089 

-0.048 

-0.130 

0.192 


0.222 

0.139 

0.191 

0.191 

0.183 

White & poverty 

-0.039 

-0.227 

0.033 

0.232 

0.012 


0.447 

0.240 

0.201 

0.240 

0.265 

White & non-poverty 

0.495*** 

0.254*** 

0.295* 

0.333** 

0.473*** 


0.121 

0.072 

0.123 

0.110 

0.104 


Notes: ***p<0.001; **p<0.01; *p<0.05;+ p<0.10 
Standard errors are in italics under the each estimate 


Further, as consistent with the analysis of math club by subgroup, non-poverty students 
tended to have higher science GPAs when in science clubs, whereas the results generally suggested 
that poverty students did not. As with math results in Table 5, Black or Hispanic and White non¬ 
poverty subgroups had higher science GPAs when also in science clubs. However, the results were 
not significant for non-poverty Asians students, though they were for math club participation. 
Again, the results for Asian students by poverty status might have been influenced by the small 
sample sizes in each subcategory. 

Effects on STEM College Major Selection 

Table 7 displays the final set of results predicting the effect of math or science club 
participation in high school on having a STEM major in college. As in previous tables, the table is 
broken out first by OLS regression models as a way to examine baseline results. However, 
propensity score matching is subsequently presented. Regardless of OLS or propensity matching 
models, the coefficients are presented as odds ratios. 
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Table 7 

Estimated E ffects of Club Participation on the Odds of Choosing a STEM Major in College 


OLS Regression Propensity Matching 


No 

Block 

Nearest 

Kernel 

covariates Full model 

Matching 

Neighbor 

Matching 


Math club 

2.766“ 

2.187* 

1.232* 

\ 404*** 

1.260“ 


0.328 

0.354 

0.082 

0.046 

0.070 

Science club 

1.606 

1.410 

1.096 

1.033 

1.112 


0.334 

0.358 

0.069 

0.042 

0.061 


Notes: ***p<0.001; **p<0.01; *p<0.05;+ p<0.10 
Standard errors are in italics under the each estimate 


Initially addressing the effects of math club participation, the first two models present the 
results from the baseline model, with the first model included no other covariates other than math 
or science club participation and the second model included the full span of covariates. The results 
indicated that participation in a math club had a positive relationship with having a STEM major in 
college. For math, the odds in the first model were almost 3-to-l that a student who was in math 
club in high school chooses a STEM major in college. The inclusion of a full set of covariates, 
however, did temper the effect math or club participation’s positive association in both subject areas 
from approximately 3-to-l odds to approximately 2-to-l odds for math club. Nonetheless, the 
coefficients remained statistically significant. 

The latter three models in the table are the results from employing three different propensity¬ 
matching schemes: block matching, nearest neighbor matching, and kernel matching. When students 
were matched based on their propensities to select into math club, there remained a significant 
likelihood that students select a STEM major in college. This is evidenced by the statistical 
significance in all three matching methods in models predicting the relationship between math club 
and STEM major selection. The odds of selecting a STEM major decreased in the propensity 
matching models; this was most likely the case because the propensity matching models were 
accounting for many factors that may have been influencing baseline estimates of club participation 
on choosing a STEM major. That being said, however, the results nonetheless suggested a greater 
probability of having a STEM major in college as a result of math club participation. 

Table 7 suggests that the results starkly differ based on club subject area; hence, the distinction 
between the effects of math and science was a crucial one. The results for science club were not 
significant, neither in baseline OLS models nor in propensity matching. The relationship between 
club participation and STEM major selection was only prevalent in math. 

Discussion 

This study has contributed a unique perspective on issues pertaining to non-formal 
experiences in STEM by evaluating the association between math and science club participation and 
STEM schooling success. This relationship was assessed using a nationally-representative sample of 
adolescent students; the dataset was a unique fit to this study because it contained student attributes, 
official school transcripts, and school data in addition to containing precise information about in 
which clubs each student had participated. Additionally, it was longitudinal in nature, thereby 
allowing this present study to evaluate concurrent STEM school success as well as longitudinal 
outcomes. 
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Overall, this study relied on two approaches. The first approach evaluated the relationship 
between having participated in a math or science club and subsequent high school (official) math or 
science GPAs. This was done by employing the data in two ways. First, a linear regression model 
was specified and was used to provide baseline results. Second, this study employed propensity score 
matching. 

The analyses in this first approach provided two major findings. First, while the use of 
propensity score matching did temper the estimates in comparison to the baseline estimates, it was 
evidenced that math club participation was related to higher cumulative high school math GPA. In 
other words, controlling for all else, the achievement gap widened in math performance between 
two identical students in which one participated in an extracurricular in-school math club and one 
did not. Second, the results indicated that science club participation was related to higher cumulative 
high school science GPA - and there was evidence presented in this study of a domain-specific 
effect. These two findings held true even after controlling for covariates previously included in 
studies on the effects of extracurricular in-school clubs (e.g., prior performance measures and 
demographic characteristics) and after controlling for new characteristics (e.g., courses taken in 
STEM and school attributes). 

A second approach provided further clarity on these two relationships. It did so by 
examining if the relationship between STEM clubs and STEM outcomes was longitudinal - i.e., how 
math or science club participation was related to the probability of having a STEM major in college. 
The baseline results suggested odds ratios that were almost 3-to-l that students who participated in 
math or science clubs would select STEM majors in college. After employing propensity score 
matching, the odds were tempered. Nonetheless, the results still identified positive, statistically 
significant relationships between math club participation and STEM major selection. 

The results in this second approach did not find any statistically significant relationships 
between science club participation and STEM major selection. However, this is consistent with prior 
research. Mathematics preparation is often used as the litmus test for measuring academic interests 
in STEM due to the fact that mathematics skills are prerequisites to participation in a large array of 
scientific and technology-based fields, such as economics, physics, and engineering (Hackett & Betz, 
1981). The same may not be true about preparation in science, as the reach of exposure in biology, 
say, may be more specialized and less generalizable to other fields in contrast to preparation in 
calculus. 

In tandem, these two approaches suggested that there was a STEM achievement gap in the 
success and persistence of students who do and do not participate in STEM-related extracurricular 
clubs. It arose from the fact that students who participate in math and science clubs tend to have 
higher math or science GPAs, respectively, during that same time-period of schooling. It also arose 
from the fact that students who participate in math or science clubs experienced a sustained 
persistence, even after high school. Therefore, math and science club participants had a 
distinguishable, positive relationship with both concurrent and future STEM outcomes. Indeed, 
these conclusions align with previous studies that highlighted factors associated with cultivating 
students' interest in math and science (Thomas, 1986) as well as with those studies that provided 
evidence of a relationship between extracurricular club participation and STEM success (Lipscomb, 
2007). 

Additional exploration, however, revealed that these results were not equal among all 
students. While, for the most part, the results were not differentiated by gender alone or 
race/ethnicity alone, they were in fact distinguishable by poverty status (i.e., whether or not the 
student’s family received a form of welfare payment) and the interaction between race and poverty 
status. This held true in both math and science regression analyses and propensity score models. 
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Indeed, the major distinction that arises between poverty and non-poverty students requires 
the need for further explanation. The results indicated that there is an overall lack of statistical 
significance on the coefficients for poverty students, and that this was true for both baseline 
regressions and propensity score models. This conclusion supports research findings that suggest 
that students from low SES backgrounds are less likely to have high academic achievement 
regardless of ethnicity (Sirin, 2005). On the other hand, students from more affluent backgrounds 
have many more support systems in place to ensure that they will succeed in school (Simpkins et al., 
2005). For example, in terms of mathematics achievement, research suggests that the home 
environment often predicts students’ attitudes and success in this domain (Sheldon & Epstein, 

2005). This is defined by how involved the parents are in terms of engaging their students in 
activities and conversation as well as showing concern for academic engagement in and out of 
school. High poverty students may experience less parental involvement compared to non-poverty 
students (Sampson & Laub, 1994). This may shed light on the mechanisms driving the results in 
Tables 5 and 6. 

Although the effects of poverty as it relates to negative academic performance can be seen 
across racial/ethnic groups, the impact is significantly greater for students in STEM minority groups 
(i.e., Black and Hispanic). Minorities are more likely to live in low-income households or in single 
parent families; their parents are likely to have less education; and they often attend under-funded 
schools (National Commission on Children, 1991). Noguera (2003) surmises that the range of 
choices available to an individual are profoundly constrained and shaped by external forces. 
Therefore, looking at the findings from this study, one might conclude that students within poverty 
may have competing external forces (i.e. resource-constrained families and schools) that prevent 
them from experiencing the full benefits of participation in academic clubs. That is, low-income 
communities and schools may have fewer activities available and fewer high-quality resources (i.e., 
teachers) to support these activities even if they were present. Moreover, youth in low income 
families may have more restraints on their free time because of caregiver and household 
responsibilities that constrain the time they can spend away from home (Simpkins et al., 2005). 

From a policy perspective, there are implications from our findings that should be 
highlighted. For the general student sample, the data showed a statistically significant association 
between STEM club participation and STEM achievement and between STEM club participation 
and subsequent selection of a STEM major in college. This can be viewed as a form of exposure that 
reinforces students’ interests and also supports their academic self-efficacy. It also further validates 
the relationship between preparation, motivation, and exposure by documenting how academic club 
participation influences academic performance in STEM (Williams & Gottfried, 2010). 

On the other hand, given the fact that there appeared to be little association between STEM 
club participation and STEM outcomes for low SES students, it would be important for further 
research to examine why it is in particular that these relationships do not hold for low SES students. 
On the one hand, it could be an institutional issue: schools serving these students might be exposed 
to lower quality teaching, fewer best practices, higher class sizes, and less access to opportunities in 
higher education — all which may inhibit students from enrolling in STEM clubs or being successful 
in STEM. Alternatively, it might be the case that low SES students are indeed interested in STEM 
but there may be external factors inhibiting successful STEM experiences, such as afterschool 
employment or family obligations. If this latter situation were the case, schools serving low SES 
students may need to be more innovative in their approach to reaching those students around non- 
formal STEM experiences. 

In thinking about ways for schools to involve low SES students, it is important to recognize 
that schools in high-poverty communities are often challenged financially which limits the available 
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resources that would support such activities. A way to combat this would be to provide targeted 
resources and supports to boost exposure to STEM-specific activities; doing so may lift the 
achievement and persistence in math and science for those youth who are at risk for educational 
failure (Sirin, 2005). This also points to the need for these schools to leverage existing resources by 
partnering with community based organizations, local STEM industries, and neighboring colleges 
and universities. These outlets of non-formal STEM exposure would build community relationships 
and create more avenues for exposure to authentic, contextual experiences in STEM for 
underserved students rather than having to rely exclusively on formal instruction or resource- 
constrained non-formal in-school activities. If these opportunities are provided in a club-like 
setting, students might then feel more compelled to participate in an ongoing basis which could in¬ 
turn have an impact on current schooling success, future college STEM participation, and ultimately 
entry into the STEM workforce. 

Limitations and Recommendations for Further Study 

There are several limitations in this study that provide points for future investigation. One 
particular limitation is that the analysis does not take into consideration the format or how the 
academic clubs are organized. While it is certain that these math or science clubs were school-based 
activities, it was not clear from the dataset whether the clubs were conducted during the school day 
(i.e., lunchtime) or after school. It was also not clear who is sponsoring the club (i.e., teacher, parent, 
administrator), nor are the qualifications of these sponsors delineated in the dataset. These details 
are important for future research: if clubs were after school rather than during school or sponsored 
by different types of individuals for low- versus high-poverty students, this may explain the 
differences that arose when evaluating the models for students by SES. 

A second limitation is the lack of specificity of the duration and consistency for which the 
students actually participated in the academic clubs. This is important to consider, as the number of 
contact hours might directly influence how well students do in their general coursework. This would 
relate back to why the data showed little association between STEM club participation and STEM 
outcomes for students affected by poverty because of the time restraints and other competing forces 
that prevent full and consistent participation. With more detail, future quantitative or qualitative 
research can focus on these precise differences between clubs in high- and low-poverty schools, 
which is not possible using the present dataset. 

A third limitation is that the study is not specific about the characteristics of the academic 
clubs, such as robotics or career interest (e.g. Junior Engineering and Technology Society). There is a 
need for further investigation of domain-specific academic clubs and activities to better understand 
how they engage students in STEM learning. To date, there is a limited body of knowledge as it 
relates to this particular subject, and it would benefit the education and policy communities to know 
how best to leverage these non-formal activities to enhance the academic achievement of all 
students. For example, robotics has played a major role in student engagement for many decades. 

But what role do robotics clubs and experiences play in STEM learning, and how do they motivate 
students to persist in STEM thus leading to STEM related academic and career pursuits? Learning 
from such activities that have established connections to both formal and non-formal settings would 
further enable a more clear understanding of how effective these activities are at substantially 
increasing STEM preparation, persistence, and success. 
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